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ABSTBACT 

An instructional program based on a learning 
hierarchy and involving the addition of fractions was presented to 
fourth grade students in a pretest - pcsttest design. The results of 
the tests nere analyzed using both Haltesser's and White's technigues 
for validation. There was sufficient disagreement in the results to 
prevent a conclusion of equivalency of the tuc methods. Attempts to 
explain the disagreements did not provide sufficient evidence to 
disregard these differences. However, a brief ccmparison of one of 
Walbesser's ratios and White's statistics did shew an interesting 
similarity, suggesting that a planned comparison might give 
statistical support to a cutoff score for the consistency ratio. 
Walbesser's procedure, with this statistical foundation, could 
provide superior information concerning both the hierarchical nature 
of the connection, and the quality of the learning program. 
(Author /BE) 
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In every curriculum construction effort, decisions must be made about how 
to formulate objectives, how to structure content, how to design and order 
instructional tasks, and how to adapt to unique student conditions . (Heimer , 1973) 
All too often these decisions are based on the philosophical belief of those 
constructing the sequence rather than an empirically developed methodology. 

In an attempt to provide a rational basis for the sequencing of content, 

Gagne, in the early sixties, initiated a program of research dealing with 

sequences of skills he termed learning hierarchies. Studies in the area have 

concentrated on three basic ideas: (1) theoretical concerns for the sequencing 

of content; (2) methods for generating the learning hierarchies; (3) methods 

for statistically supporting the generated hierarchies. Numerous statistical 

models have been proposed and supportc^d by various philosophical and 

statistical arguements. Many of these methods have weaknesses ranging from 

slight to severe. Two of these techniques. White's and walbesser's, hold 

particular promise and are the subject of this study. 
# 

Sequencing Content 

With regard to content sequencing, Briggs stated that "for learning 
purposes, then, 'structure' means ... the description of the dependent and 
independent relationships among component competencies, arranged so as to 
imply when sequencing can be random or optional and when sequencing must be 
carefully planned, on the basis that transfers will be optional to build up 
froxu simple skills to r-jre complex ones." (Briggs, 1968) 
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In the early sixties, Gagne took a conmion sense approach to the problem 
of structuring content. Beginning with a desired outcome. Gagne asked the 
question. "What capabilities would an individual have to possess if he were 
able to perform this task successfully given only instructions?" The answer 
to this question would be a new set of tasks, distinct from, but prerequisite 
to the original task. According to Gagne the derived tasks are "in some 
sense simpler and more general" than the original . (Gagne. 1962 ) The process 
of questioning and deriving new tasks is applied to each of the tasks derived 
from the original task. Gagne found that by repaeting this process, he was 
defining a hierarchical structure, corresponding to Briggs' hierarchical 
structure, that was growing progressively simpler and more general. The 
hierarchical structure defined by this process is termed a learning hierarchy 
and represents a series of proposed dependencies among a set of intellectual 
skills. 

Viewing the sequencing of content in the context of learning hie. 'archies. 
Ggane wrote." I am not sure that a learning hierarchy is supposed to represent 
a presentation sequence for instruction in an entirely uncomplicated way. 
Presumably, there should be some relation between an ordered set of intellectual 
skills and an ordering of a sequence of presentation ... in an instructional 
program."(Gagne. 1968 ) Others however have not been quite as hesitant as 
Gagne on the relationship between hierarchies and content sequencing. Okey 
said. "A hierarchy of tasks ... can serve as an instructional map or guide 
frr . teacher because it is a list of which tasks are to be accomplished by 
students as well as the sequence in which they should be studied and 
taught. "(Okey) Briggs in discussing the learning hierarchy presented in the 
Gagne and Paradise article stated that. "These subordinate competencies ... 
must be taught in a particular sequence-, (with some options within layers of 
the pyramid) rather than in a random sequence. "(Briggs. 1968) Passmore in 
discussing objective-based instructional systems, listed three purposes for 
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learning hierarchies: 

(1) They describe the most efficient way to sequence instruction. 

(2) They serve as a guide for the student's entry into a learning sequence. 

(3) They serve as a tool used in criterion-referenced assessment plans for 
monitoring student progress through objective-based instructional 
systems. (Passmore) 

Obviously, some school learning involves the memorizaj:ion of independent 
facts, for such cases the sequence of presentation is of no consequence. 
Hopefully, however, much of the learning in school involves attainment of 
interrelated skills the learning of which facilitates the learning of other 
higher order skills. For this type of learning, a learning hierarchy probably 
does represent a valid sequence of instruction. Support for this statment 
can be found in a number of studies. Walbesser and Eisenberg, in an 
excellent summary of research, found that a majority of studies supported the 
following hypothesis: The aacquisition of a terminal behavior depends upon 
the attainment of a hierarchy of subordinate behaviors that mediate positive 
transfer from one set of behaviors to the next higher relevant behaviors in 
the sequence and eventually to the terminal behavior. (Walbesser, 1972) 

Generating Hierarchies 

To generate hierarchies, suppositions must be made about what constitutes 
a valid element of the hierarchy and how they are to be ordered. Gagne used 
the term "learning hierarchy" to refer to a set of specified intellectual 
capabilities having an ordered relationship to each other. Gagne, in discus- 
sing these sets wrote that these capabilities are vhat some writers call 

"cognitive strategies What they are not is just as important. They are 

not entities of verbalizable knowledge. I have found that when deriving them 
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one n>ast carefully avoid statements of 'what an individual knows'." (Gagne. 1968 ) 
Smith gave three reasons for thinking of the elements of a learning hierarchy 
as performance requirements rather than as simply elements of verbalizable 
knowledge. First, since presentation of information does not imply assimilation 
of that information, some actual evidence must be specified to assure th'at 
the desired learning has taken place. Second, since recall of information 
is seldom justification for instruction, capacity for using that information 
to organize new information is a more useful outcome. Third, it is quite 
likely that more than presentation of verbal statements is required to develop 
capacity to utilize information. (Smith, 1972) 

Once the character of elements which constitute the hierarchy is" 
determined, a procedure must be chosen for generating the hierarchical structure. 
Passmore listed four methods for generating hxerarchies: (D observation. 
(2) formal analysis, (3) statistical fishing, and (A) introspection. (Passmore) 
Gagne's method of generating hierarchies, through questioning, is an example 
of introspection. The technique involves subject matter experts using their 
knowledge about the structure of their area and the nature of learning in 
their discipline. Using the process, the resulting hierarchy is somewhat 
dependent on the knowledge of the experts who developed the structure. To 
insure that a truely hierarchical ordering has been achieved, a validation 
procedure is needed. 

Validating Hierarchies 

Theoretically, a 'valid' connection in a hierarchy implies that if the 
lower tasks are not mastered then the ones above them cannot be. A validated 
hierarchy then is one which consists of only valid connections between skills. 
Connections to be validated are examined as to the degree the performance of 
learners indicates compliance with the definition of a valid hierarchy. 
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Performance data can be gathered in essentially two ways. First, learners 
may be given a test consisting of questions on each of the skills in the 
hierarchy- The patterns of right/wrong answers can then be analyzed and the 
degree of agreement with the definition of hierarchical structure can be 
determined. Second, learners may be tested on their knowledge of each of the 
skills of the hierarchy during or immediately following instruction in the 
skills from the hierarchy. While practically all the validation techniques 
developed so far can use either type of data, various researchers have put 
forth arguements to support their use of one method or the other. 

Al though "Tiiany different techniques have been developed for validation 
since Gagne initiated research into learning hierarchies, two are of 
particular importance as they represent basic modifications of Gagne's 
original efforts. Walbesser and Eisenberg expanded Gagne's original index 
of positive transfer into a series of 5 indexes . (Walbesser , 1971) Based on 
the performance data for both the higher and lower skill of a connection, 
the subjects are divided into four groups indicated by the following 
ordered pairs: 

(0,0) which indicates failure on both the lower and higher level skills. 

(0,1) which indicates failure on the higher level skill while passing 
the lower level skill. 

(1.0) which indicates success on the higher level skill while failing 
the lower level skill. 

(1.1) which indicates success at both levels. 

The indexes were then defined in terms of the number of subjects in each group. 
In the ratios below f(l,l) represents the number of subjects in the (1,1) group. 
Of the five indexes, three are of immediate interest in hierarchy validation. 

(1) Consistency Ratio - This ratio measures the strength of the implication 
that acquisition of the terminal behavior implies acquisition of all sub- 
ordinate behaviors. 

f(l.l) 
f(l,l) + f(l,0) 



(2) Adequacy Ratio - This ratio tests the strength of the implication 
that acquisition of all subordinate behaviors implies, with instruction, 
acquisition of the terminal behavior. 



f(l.l) 



f(l.l) + f(O.l) 

~~ (3) Completeness Ratio - This ratio estimates the percentage of 
individuals capable of traversing the hierarchical connection in question. 

f(l.l) 

f(l,l) + f(O.O) 

A connection is considered validated if each of these 3 ratios has a value 
of 0.85 or above. 

This is an important extension of Gagne's measure of validity. The ratios 
not only measure the negative side, that a learner cannot learn the higher 
task without the necessary lower order skills; it also measures the positive 
side, that having mastered the lower order skills. the student is more likely 
to achieve the higher level tasks. The consistency and adequacy ratios 
measure whether the connection is consistent with the hierarchical hypothesis 
while the comrleteness ratio measures whether a sufficient percentage of 
subjects were able to do either task and thus be included in the statistics. 

White listed eight criticisms of the model for learning hierarchy valida- 
tion used by Gagne. (White. 197A ) One of White's objections was the absence 
of an index based on a standardized distribution. As part of a new model 
for hierarchy validation. White and Clark developed a statistic based on the 
binomial distribution. (White. 1973 ) Through the use of the statistical 
distribution. White's method takes into account errors of measurement 
associated with the testing of the skills. Although a complete description 
of White's method can be found in the previous reference, basically if two 
questions are being used to measure acquisition of each skill, the number of 
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subjects who correctly answer both questions for the higher skill and miss 
both questions for the lower skill becomes the critical frequency. Using 
White's formulae, an expected frequency for this group is calculated. If 
the expected frequency exceeds the observed frequency then the connection is 
considered validated. While White's method is the more rigorous of the two 
methods discussed, it is much more difficult to use. Walbesser's model is 
relatively easy to use but lacks the statistical rigor of White's model. If 
a correspondence could be established between the two methods, then the use 
of the less complicated technique could be justified in terms of the the 
statistical validity of the other. The purpose of this study was to inves- 
tigate the possibility of such a correspondence. 

Experimental Design 

An examination of Walbesser's and White's models shows that the chief 
point at which the two techniques differ is the statistic used to either 
support or reject the hypothesized connection!, in the hierarchy. Thus, a 
learning program based on a learning hierarchy was presented to a single 
sample of students. The results of the associated tests were analyzed 
using both Walbesser's and White's techniques. Where the techniques led to 
different conclusions, attempts were made to explain these differences. 

For the purposes od a study of validation techniques, Kane and others 
developed a proposed hierarchy on the addition of fractions. A programmed 
booklet was written to teach the skills of the hierarchy. Kane also used 
two pretests to measure the level of prepareciness of the subjects and 
posttests to measure the achievement of the skills. The first pretest was 
used to insure that students included in the study had the background 
necessary to begin the hierarchy. The second pretest was used to eliminate 
students who already had mastered a majority of the material. For the purposes 
of this study, the programmed booklet was modified to include test frames 
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for each skill. The posttest consisted of these test frames and no separate 
testing was done. Since it is important to know if a skill is available 
at the time it is needed to provide prerequisite knowledge, questions concern- 
ing a skill were placed immediately preceding the instruction for a skill for 
which the first skill was considered prerequisite. 

The subjects in the study were fourth grade students who possessed the 
necessary prerequisite knowledges as evidenced by a score of 50% or better 
on Pretest I and who had not mastered a majority of the material as evidenced 
by a score of not more than 50% on Pretest II. The students spent approx- 
imately thirty minutes per day working through the booklet. The performance 
data was then extracted by the experimenter by grading each test frame. 
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Data Analysis 

Two questions were used to measure the achievement of each skill. Thus 
performance data, for each connection consisted of right/wrong responses from 
2 questions for the lower skill and 2 questions for the higher skill. Subjects 
were then divided into nine groups based on 0,1,2 correct answers for the 
lower skill and 0,1,2 correct answers for the higher skill. The data for 
each connection was recorded in a 3 by 3 table. 



Higher Skill 
0 1 2 



Lower 
Skill 



0 
1 
2 



For White's method, the critical frequency was the number in the cell 
representing no correct responses on the lower skill and two correct responses 
on the higher skill. Using White's formulae and the data in the tables, an 

xpected frequency was calculated for each connection. If the expected 
frequency exceeded the observed frequency, the connection was validated. 
If it equals the observed frequency, then judgement about the connection is 
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suspended. If the expected frequency is less than the observed then the 

rtion is rejected. The results of these calculations are shown in Table I. 



connect 



Table I 



Analysis of Connections By l-Jhite's Criteria 



Connection 



Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 



I to Level lib 
I to Level lie 
Ila to Level Va 
Ila to Level 



III 
III 
IV 

Ild to Level IV 

III to Level IV 

IV to Level Vb 
Va to Level VI 
Vb to Level VI 
VI to Level VII 



lib to Level 
lie to Level 



Expected 
Value 

3 
4 

41 

31 

2 

8 

9 

5 

7 

4 

12 
14 



Observed 
Value 

13 
13 
76 
72 
16 
3 

18 
5 
5 
3 

16 
37 



Validated 

No 

No 

No 

No 

No 

Yes 

No 

Suspend Judgement 
Yes 
Yes 
No 
No 



While \>/hite's technique can handle the case where a subject answers only 
one of the two questions correctly, Walbesser's method requires that a 
judgement be made as to whether or not a student has acquired the particular 
skill. For the purposes of this study, a student was judged to have acquired 
a skill if he answered at least one of the two questions correctly. The 
data in the 3 by 3 tables was reduced to 2 by 2 tables. For each- connection, 
the consistqficy , adequacy and completeness ratios were calculated. The 
results of this analysis are shown in Table II. The results of the two 

procedures are summarized in Table III. Of the twelve connections. White's 
procedure validated three and suspended judgement on one. Walbesser's method 
failed to validate any of the connections. 



Conclusions 

Prior to the start of this study it was agreed that the two methods 
would be considered equivalent if no more than two disagreements were found. 



Table II 

Analysis of Connections By Walbesser's Criteria 



Consistency 



Connection . R^^iQ 

Level I to Level lib .83 

Level I to Level lie .73 

Level Ila to Level Va -23 

Level Ila to Level III .19 

Level lib to Level III .79 

Level lie to Level IV .87 

Levei. I Id to Level IV .37 

Level III to Level IV -87 

Level. IV to Level Vb .83 

Level Va to Level VI .84 

Level Vb to Level VI .51 

Level VI to Level VII .57 



uacy Completeness 



R atio Ratio^ Validated 

.84 .73 No 

.67 .68 No 

.74 .46 No 

.78 .29 No 

.77 .74 No 

.79 .58 No 

,70 .36 No 

.69 .64 No 

.54 .43 No 

.59 .71 No 

.75 .41 No 

.72 .69 No 



Table III 
Comparison of Evaluated Connections 



Connection 



Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 
Level 



I to Level 
I to Level 
Ila to Leve 
Ila to Leve 
lib to Leve 
JIc to Leve 
I Id to Leve 

III to Leve 

IV to Level 
Va to Level 
Vb to Level 
VI to Level 



lib 
lie 



Va 

III 

III 

IV 

IV 

IV 

Vb 

VI 

VI 

VII 



Results of 
Wiite's Method 

No 
No 
No 
No 

No 

Yes 

No 

Suspend Judgement 
Yes 
Yes 
No 
No 



Results of Agree/ 

Walbesser's Method Disagree 

No Agree 

No Agree 

No Agree 

No Agree 

No Agree 

No Disagree 

No Agree 

No Disagree 

No Disagree 

No Disagree 

No Agree 

No Agree 
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There Is sufficient disagreement ir. tha data of this study to p. event a 
conclusion of equivalency. The results of using Walbesg^r's formulae with 
this hierarchy differ markedly from the results obtained by Kane. However, 
Kane restructured connections which were not validated arid then recalculated 
the results to obtain an optimum ordering. Caution ^houJd be exercised in 
using this approach or a hierarchical relationship between two unrelated 
topics may result not unlike a high correlation between 2 unrelated measures. 

Attempts to explain the disagreements do not provide sufficient evidence 
to disregard any of the differences. Therefore, the two methods should be 
examined to find any differences that may exist. A look at connection nine, 
between Level IV and Level Vb can illustrate some of the^e differences. For 
White's method, the expected frequency was 5 while t^e ol^served frequency 
was 7. Using Walbesser's method, the consistency ratio vaS .82, the adequacy 
ratio was .53 and the completeness ratio was .36. 

Whether a hierarchy is validated using a learninj^ program Specifically 
designed to teach the skills of the hierarchy or usit^g d^ta based on prior 
learning, a hierarchy must be associated with some learning experience. A 
close look ay the meanings of Walbesser's ratios and their numerical values 
for this connection gives valuable information about the relationship between 
the two skills and the related instructional sequence- consistency ratio 

Is designed to measure the degree to which the conne^tiofl under scrutiny is 
consistent with the theory of hierarchical relationships. For this connection, 
the consistency ratio was .82. This indicates that 1 82 <?f the subjects 
exhibited behavior contrary to the hierarchical hypothesis- Although 
Walbesser set a cutoff of .85, the .82 ratio certainly g^^es support to the 
hierarchical hypothesis for this connection. 
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The adequacy ratio determines whether the elements of the hierarchy 
adequately define the skills necessary to achieve the higher skill. For 
connection nine, the adequacy ratio was .53. Rather than saying the relation- 
ship is not hierarchical, it indicates that A7% of the subjects who obtained 
the lower skill could not obtain the higher skill. This points to a need for 
either additional elements inbetween these two or to a need for improved 
instruction for the higher skill. 

The completeness ratio measures the proportion of the available subjects 
who have achieved any of the skills involved in the connection. For 
connection nine, the completeness ratio was .36. This implies that 6A% of 
the subjects achieved no skill in the connection. Thus any decision made 
on the validity of this connection is based on less than half the subjects. 

l^ite's method specifically tests the hypothesis that the proportion of 
persons with the superordinate skill only is .ero. For connection nine, the 
expected frequency was 7 while the observed frequency was 5; and thus the 
connection was validated. Although the hierarchy can not be separated from 
the learning experience. White's formulae give no distinction between the 
validity of the hierarchical hypothesis and the effectiveness of the learning 
program. 

It is interesting to note that this connection which was validated by 
White's method also had a consistency ratio of .82. In fact each of the 
connections validated by White's method had high consistency ratios. The 
calculation of the expected frequency for the critical cell involves the 
frequencies in all the cells - the table and thus the calculation is effected 
fby the quality of the instructional program. While both methods are effected 
by the instructional sequence. Walbesser's method gives more detailed information 
which can be used to either revise the hierarchy or improve the instruction. 
While White's technique uses statistical rigor for validating the hierarchical 
nature of the connection, it gives no additional information which could be 
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of value in revising the hierarchy. Because of the similarity between 
information from White's formulae and Walbesser's consistency ratio, the 
establishment of a cutoff value for the consistency ratio using the 
statistical basis of l^hite's technique could be useful. 

Although the data of the study did not support a claim of equivalency 
for the two methods, a brief comparison of onc^ of Walbesser's ratios and 
White's statistics shows an interesting similarity. A planned comparison 
of these two might give statistical support to a cutoff score for the 
consistency ratio. Walbesser's procedure with this statistical backing 
would provide superior information concerning both the hierarchical nature 
of the connection and the quality of the learning program. 
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Learning Hierarchy for Addition oi fractions 
with like Denominators 



Level VIII 



Level VII 



Level VI 



Level V 



Level IV 



Level III 



Level II 



Level I 





Addition of Rational 
Numbers with like 
Denominators 












Sum of 3 numerals - requiring 1 
reducing and renaming j 







Sum of 2 numerals - requiring 
reducing and renaming 



Renaming Improper 
Fractions 3 



Sum of 3 mixed nu^nerals 
Sum needs reducing b 



Sum of 2 mixed numerals 
Sum needs redu cing 



Sum of 2 fractions 
Sum to be reduced 



Reducing to 
lowest terms a 



Sum of 3 fractions 

in lowest ternii L 
X ■ 




Sum of 2 
mixed numbers 



Sum of J 
fraction and 
whole nurjbar 



Sum of 2 fraccions 
in lowest teri-ns 



Learning Program 



Prerequisites 



tyhole Number 
Arithmetic 



Basic Concept 
of Fraction 
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